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This paper describes the development of the Test of Science Processes, an 
instrument for assessing achievement and diagnosing weaknesses in the use of 
science processes by students in grades 7, 8, and 9. The science processes 
considered were observing, comparing, classifying, quantifying, measuring, 
experimenting, inferring, and predicting. The test consists of 96 multiple-choice 
questions and requires an actual testing time of 73 minutes. Scoring of the instrument 
yields a total score and eight sub-scores, one for each process. The test was initially 
administered to 3,673 students in the state of New York representing all ranges of 
ability and socio-economic background. Norms are reported by grade, sex, urban 
versus suburban and for the total sample within each grade. There is evidence of 
both content and curricular validity of the test. There is also evidence for criterion 
related validity, but more conclusive evidence is needed. The predictive validity of the 
test has yet to be studied. (BC) 
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SUMMARY 

In this study, an Instrument to assess achievement and diagnose 
weaknesses In the use of science processes by students in grades seven, 
eight, and nine was developed. It is entitled Test of Science Processes 
The science processes considered were: Observing, Comparing, Classify- 

ing, Quantifying, Measuring, Experimenting, Inferring, and Predicting. 
These were arrived at after a study of relevant texts and other liter- 
ature. The original list was edited and condensed and the behaviors 
which students must exhibit in order to demonstrate competence in the 
use of each of the processes were specified in detail. This state- 
ment of behaviors was submitted to experts for their opinions and 
validation before final writing. The final version of the statement 
of behaviors served as the blueprint for the Test of Science Processes . 

The Instrument has the following characteristics: 1. it consists 

of ninety-six multiple choice (five choice) questions; 2. it requires 
total actual testing time of seventy-three minutes (some students may 



I finish in less); 3. the test booklet is printed with black and white 

Illustrations and, for the twelve questions xdilch require color, 35 mm 
color slides are used; 4. scoring of the Instrument yields a total 
score (Kuder-Richardson formula 20 reliability “ .91) and eight sub- 
I scores, one for each process (reliabilities from about .30 to about .80) 

The Instrument was administered to 3,673 students from schools in 

the Bronx, New York, and Rockland County, New York. This sample was 

I 

\ carefully selected to include all ability levels and a wide range of 

socio-economic backgrounds. The results of this administration were 
used to create norms for total score and for each of the el^t sub- 
i scores. The norms are reported by grade, by sex, and by urban verses 

I 

I suburban, and for the total sample within each grade. The results were 
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also employed ir a study of the unl<}ueness of each of the subscores 
(l.e«, the proportion of the non'^^srror variance which is unique to 
that subscore.) Six of the subscores were found to have uniquenesses 
between .1244 and .1578. The other two subscores had uniquenesses 
which are not significantly different from ?,ero but this may be at- 
tributable to their short lengths. The item analysis data tor ali 
ninety— six questions administered to these students are available. 

There is considerable evidence of both the content and the cur- 
ricular validity of the test. The criterion— related validity of the 
test is very difficult to assess because this is the first attempt to 
measure this age level students' ability to use science processes j A 
small investigation was undertaken to demonstrate the criterion-related 
validity (where the criterion is teacher rating of the students' 
abilities to use the processes.) The results of tnis investigation 
were not unequivical. However 9 they do give some indication of a 
degree of criterion-related validity. More conclusive evidence of 
this must await much more extensive investigations. The predictive 
validity of the Test of Science Processes has not yet been studied. 



ACCOUNT OF CONSTRUCTION 



It was decided that the test would concentrate exclusively on 
assessing students' ability to use science processes. This approach 
proved to have advantages. It allowed for the production of a test 
which draws its examples from several of the natural sciences and 



is shorter that is normally necessary for assessing achievement 
in science" based on traditional subject demarcations. For this 
reason, it is thought that different methods of teaching may affect 
students' performances on the test, but different subject matter 



r emphases should not. This remains to be Investigated. 
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The first major step to be undertaken was the definition of and 
statement In behavioral terms of those science processes which It was 
appropriate to expect seventh, eighth, and ninth graders to be able 
to use in the light of current science education. This statement 
was meant to serve. In its final form, as the blueprint for the Test of 
Science Processes . 

Many elementary and junior high school science texts and research 
reports were carefully studied and summarized to determine the processes 
ol science which seventh, eighth, and ninth graders are currently 
expected to be able to use. The resulting list of processes was then 
carefully reorganized, condensed, stated in behavioral terms, and 
edited into a more polished statement which was ready for validation. 

The behaviors on the list were categorized under eight processes: 
Observing, Comparing, Classifying, Quantifying, Measuring, Experimenting, 
Inferring, and Predicting. 

Evidence of the curricular validity of the statement of science 
processes was gathered by submitting it to experts in science education. 
The statement was mailed to about thirty prominent science educators 
(these are listed in Appendix A of this report along with the require- 
ments for inclusion on this list and the instructions for responding to 
the statement.) 

Twenty-tv7o usable responses were returned and, on the basis of 
these, the final statement of processes was written. It was decided 
arbitrarily that the following rules would be adhered to in writing the 
final statement: 

1. No new behaviors would be added to the final statement of 
processes . 

2. No behavior would be Included in the final statement of 
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processes which more than one-third of the respondents decided 
was Inappropriate with regard to difficulty (either too hard 
or too easy.) 

3. No behavior with a mean of less than 2.5 on the three-point 
"appropriateness in the light of current science education" 
scale would be included in the final statement of processes. 

4. All behaviors listed in the statement submitted far validation 
which passed rules two and three, above, would be included 
verbatim in the final statement of processes unless their 
mean scores on the three-point "clarity" scale were below 2.5, 
in which case they would be given minor rewriting in accordance 
x<7lth the suggestions of the respondents before inclusion. 

Only one behavior was rejected as too difficult and only three 
were considered inappropriate In the light of current science education. 
Six of the remaining fifty-three behaviors were rewritten In accordance 
with the suggestions of the respondents because their mean scores on 
the "clarity" scale were less than 2.5. (None of these six had a 
mean of less than 2.33.) 

The final statement of processes which was produced and used as 
the blueprint for the Test of Science Processes thus consists o.f 
fifty-three behaviors categorized under eight major processes of 
science. Of these, forty-seven were validated verbatim by the science 
education experts and six were validated with minor rewriting suggested. 

On the basis of the blueprint. Form I of the Test of Science 
Processes was written. Form I consisted of ninety-eight multiple-choice 
items. Each Item consisted of a 35 mm color slide visual stimulus, a 
mimeographed stem, and five choices. Form I was designed to be used 
with simultaneous vocal stimulus, also. The voice gave instructions 
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and read the questions to mlniaize reading problems. 

Form I was administered to 156 seventh, eighth, and ninth graders 
from a Bronx, New York, public Intermediate school In January, 1968. 

The results of this administration were carefully Item analyzed and, 
on the basis of the analysis. Form II of the Test of Science Processes 
was written. 

Three items were dropped from Form I and one new Item was added 
so that Form II consisted of ninety-six multiple choice Items. Many 
of the other questions from Form I were given at least some rewriting 
prior to Inclusion In Form II on the basis of the Iteri analysis and 
the suggestions of Dr. Willard J. Jacobson and Dr. Robert L. Thorndike, 
of Teachers College, Columbia University. 

The ninety-six Items and printed Instructions were typewritten 
(with right-hand justified margins) and the color slides (except 
those for questions one through twelve on Form II) were converted to 
tables or black and white prints. The correct answers to the items 
were randomized using a table of random numbers and the questions 
were arranged in a *’sax<7-toothed" order of increasing difficulty within 
each subscore (based on the difficulty of the item when Included on 
Form I) . The entire test was then ’’cut and pasted” into ’’camera-ready 
copy to be photo offset and commercially printed. 

After Form II was printed, it was administered In February, 1968, 
to 3,673 seventh, eighth, and ninth graders from Rockland County, New 
York, and the Bronx, New York. The results of these administrations 
were used to determine the norms. Item statistics, reliabilities, and 
validities . 

Following the administration of Form II, the statistical analyses 
were performed using the facilities of the Teachers College Computer 
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Center. The final step in the construction of the Test of Scie nce 
Processes was the writing of a very complete manual and the inclusion 

in it of the statistical tables. 

The procedures for construction of this type of test are fre- 
quently enumerated (if less frequently employed) and I shall assume 
that this audience is well acquainted with them. Therefore, I shall 
devote the rest of this paper to those aspects of the Test of Science 
Processes and its development which are somewhat unique. 



BLUEPRINT 
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Prior to the actual construction of items and the arrangement of 
them into a test, a blueprint must be created which will provide guide- 
lines for the preparation and contents of the final instrument. 
Traditionally, blueprints are two dimensional with content areas 
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specified along one axis and the behaviors to be performed specified on 
the other axis. At each intersection of behavior and content area, 
the number of items (or, sometimes, the percent of the total test) 
which will deal with using that behavior on that content is specified. 






The blueprint for the Test of Science Processes is essentially 
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one dimensional. In this sense, it is rather untraditional . It was 
decided very early in the construction of the test that it would be 
concerned strictly with assessing students* abilities to use science 
processes regardless of particular science content. Therefore, one 
dimension is simply "science.” It was the other dimension - the science 
processes — which had to be carefully and specifically defined in 
behavioral terms. 

The definition of those science processes to be Included Involved 
several stages. The first stage was a detailed survey of the relevant 
literature. This included a careful investigation of major statements 
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of the psychology of science learning and of the philosophy of science 
and science education. In addition to theoretical sources, numerous 
science tests, tesi^ts, and syllabi were consulted. All of the liter- 
ature which was surveyed is listed in Appendix B of this paper. 

There are clear parallels between the AAAS list of processes and 
the blueprint of the Test of Science Processes . Indeed, perhaps the 
greatest single debt is owed to this one source even though, as x^ith 
all the other research and literature reviewed. It is not specifically 
directed at seventh, eighth, and ninth graders. And, while It Is very 
similar to and was vary helpful in creating the blueprint, many other 
sources were also consulted in order to produce a valid and workable 
blueprint . 

In summarizing the extensive review which was done of the liter- 
ature relative to defining science processes which students of this 
age group should be able to use. It may be said that while much work 
has been done on similar problems and closely related topics, there 
Is nothing specifically defining processes of science which it is ap- 
propriate to expect seventh, eighth, and ninth graders to be able to 
use. Therefore, the task remained to refine what had been done before, 
combine it with what is currently expected of students in these grades 
as evidenced by current tests, texts, syllabi, etc., and synthesize 
and validate a blueprint. The final result of this undertaking - the 
blueprint of the Test of Science Processes - is Included as Appendix C 
of this paper. 

SAMPLE TEST ITEMS 

All of the Items on the Test of Science Processes are five choice, 
multiple choice items. Each was written specifically to test one of the 
abilities detailed In the blueprint. To give a brief sample of the 
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test» several Items are Included in Appendix D of this paper* 
RELIASyxITY 

7i'he following table presents reliability data for the Test of 
Sci ence Processes , Form II. The reliabilities reported are the results 
of internal analyses (using the Kuder“*Richardson Formula 20) based on 
a single administration of the test* Correlations between alternate 
forms of the test and test-retest correlations have not been obtained* 

The sample is the same as that used for normlng the test* It was 
drawn from public junior high schools in Rockland County 9 New York, 
and in the Bronx, New York* 

Although the test is timed during administration, this probably 
had a negligable effect on the performances of the students because? 

(1) the time limits were liberal enough to allow most (over 84%) of the 
students to finish, ( 2 ) all students were forced to attempt all parts 
of the test, and ( 3 ) if they finished early, students were allowed to 
go back and work on any question (s) which they skipped either because 
of difficulty or time limitations in a particular section* 



TABLE I 
RELIABILITIES 


SUBURBAN — 
7 8 


GRADE 

9 


URBAII 

7 


8 


GRADE 

9 


TOTAL 

7 


8 


GRADE 

9 


TOTAL SCORE 
96 ITEMS 


*91 


*91 


*91 


*90 


*91 


*90 


*91 


*91 


*90 


I OBSERVING 
9 ITEMS 


*47 


*41 


*47 


*41 


*42 


*41 


*41 


*42 


*42 


11 COliPARING 
5 ITEMS 


*34 


*35 


*26 


*37 


*31 


*31 


*37 


*31 


*30 


III CLASSIFYING 
13 ITEMS 


*71 


*67 


*62 


*62 


*58 


*60 


*64 


*60 


*60 


IV QUANTIFYING 
12 ITEi:S 


*74 


*75 


*69 


*69 


*67 


*64 


*69 


*68 


*65 


V MEASURING 
25 ITEMS 


*79 


*79 


*82 


*71 


*80 


.77 


*72 


*80 


*78 j 


VI EXPERIMENTING 
10 ITEMS 


*54 


*54 


*57 


*43 


*47 


.46 


*45 


*49 


.48 


VII INFERRING 
14 ITEMS 


*53 


*48 


*59 


*60 


*63 


*57 


*59 


*62 


*57 


VIII PREDICTING 
8 ITEMS 


*32 


*42 


,51 


*49 


*56 


*48 


*48 


*55 


*48 







-9 



VALIDITY 

The validity of an instrument such as the Test of Science 
Processes is a crucial question. There is considerable evidence of 
both the content and the curricular validity of the test, due to the 
care employed in the construction and validation of the blueprint and 
items. The technique employed is often referred to as "validation by 

experts." 

The predictive validity of the Test of Sci ence Processes has not 
yet been investigated because there has not yet been time enough since 
the creation of the test to follov? students through junior and senior 



high school and correlate high school achievement with results of the 
test. At least until these data can be gathered (and even thereafter) 
it is urged that the results of the Test of Scj^n cePrcy esses be used 
as only one of several sources of infornation employed by professional 
guidance personnel to help students plan courses of study and remediation, 
The criterion-related validity of the Tea^f_Jcl^^^^ 
is extremely difficult to investigate because this is the first at- 
tempt to assess this age level students* ability to use science pro- 
cesses. Although it may seem slightly irreverent to make the comparison. 



the problera is somewhat akin to the problems inv<?ived in validating 



the first XQ test at the beginning of this cent::ry. 

In an a.r.Cempt to provide at least some evidence of criterion- 
related validity, a small investigation was undertaken. A group of 
students (N « 35) took the Test of Science Processes^ and their 
teaclior, who had obser^/ed their classroom and laboratory behavior for 
more than a full semester, rated them individiiaUy on a scale of zero 
to nine on each of the eight processes. The tea her was gf/en a copy 
of the blueprint and carefully instructed to rate each of his students 
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-ic- 
on his or her ability to use the processes as defined therein. In an 
attempt to minimize "halo effect" and increase the reliability of the 
teacher's ratings, the teacher rated each student on the first process 
and then shuffled the papers so as to rate the students in a different 
sequence on the next process and so on through the eight processes. 

The students' eight raw subscores on the test were correlated with 
their teacher's eight ratings. The students' total raw scores were 
correlated with the sum of the teacher's eight ratings. High cor- 
relations in the principal diagonal of the resulting nine by nine cor- 
relation matrix would be evidence of validity related to the criterion 
of teacher rating of ability to use the processes. 

, Clearly there are weaknesses in this investigation (e.g., small 
sample, p^jsible teacher biases, untested nature of the rating 
system) . However, in view of the unique nature of the Test of Science 
Processes , this was considered a small price to pay for an indicator 
of validity in the absence of other more traditional tech 'iiques . 

The results of the correlation analysis are given in Table II 
on the following page. 

In Table II, the correlations of particular interest are those 
along the principal diagonal (they are heavily underlined) . These 
are the correlations between the teacher's ratings and the students' 
subscores and total scores. The three bottom rows contain the rank 
of the correlation in the principal diagonal relative to the other 
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correlations in that column (these could range from one to nine) , 
the average of all of the correlations except those in the principal 
diagonal in each column, and the difference between the correlation 
in the diagonal and the average correlation in each column. 

As evidence of the criterion-related validity of the Test of 
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Science Processes » the following should be noted: 

1. In all but one case, the correlation of the teacher's rating with 
the students' subscores was higher than the median correlation in that 
column (rank of four or better); 2. in all but one case, the cor- 
rel<^tlon in the principal diagonal was higher than the average of 
all the other correlations in that column; 3. in all but two cases, 
the correlations in the principal diagonal ranked second or third in 
their columns, once fourth and once sixth; and 4. all of the cor- 
relations in the principal diagonal are positive and six of them are 
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Significantly different than zero, two of these with probability less 
than .01. 

Clearly, this evidence is not unequivlcal. One should not expect 
it to be so in view of the low reliabilities of the short subscores 
and the imperfections of the experiment (small N, teacher bias, etc*)* 
However, it may be interpreted as at least some indication of a 
degree of criterion-related validity where the criterion is teacher 
rating of the students' ability to use the processes* 

Several other much more extensive experiments of this and 
similar natures are contemplated to further investigate the criterion- 
related validity of the Test of Science Processes * Investigations 
of some of the correlates of success on the test are also under con- 
sideration* As these are completed, they will be reported* 
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CORRELATIONS 

I 

For the use of researchers and others who may be interested, j 

2 

the product-moment correlations of the total score and the eight 
subscores for the entire normlng sample are presented below* Since 
it may be desirable to study data for only a particular subgroup of 
the population, the raw data for calculating other sets of correlation 
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coefficients is available from the author in punched card form. When 
referring to these correlations, users are cautioned (1) to consult 
the corresponding reliabilities since these dictate the theoretical 
maximum meaningful correlations and (2) to keep in mind that the 
correlations between subscores and the total score are artificially 
raised by inclusion of the subscore within the total score. 



TABLE III CORMXATION MATRIX 



PROCESSES (subscores) 
I II III IV 


V 


VI 


VII 


VIII 


TOTAL SCORE .666 


.586 .758 


.786 


.883 


.685 


.740 


.665 


I OBSERVING 1.000 


.397 .484 


.481 


.524 


.379 


.405 


.346 


II COMPARING 


1.000 .489 


.471 


.425 


.353 


.341 


.310 


III CLASSIFYING 


1.000 


.584 


.587 


.437 


.452 


.389 


IV QUANTIFYING 




1.000 


.652 


.438 


.461 


.441 


V MEASURING 






1.000 


.536 


.564 


.511 


VI E30PERIMENTING 








1.000 


.497 


.423 


VII INFERRING 










1.000 


.557 


VIII PREDICTING 












1.000 


UNIQUENESS 














In a test such as 


the Test of Science Processes 


that 


contains 





subscores which are unique in a logical and a curricular sense, the 
question must be asked if they are also unique in an empirical sense. 
The first investigation of this question which was undertaken was a 
principal components factor analysis of the matrix of intercorrelations 
of the subscores. This analysis is summarized in Table IV. 

The analysis was first performed with unity in the principal 
diagonal and the result was one gereral factor, about evenly weighted 
on all eight subscores, which accounted for better than fifty percent 
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or the variance. A second factor was found which accounted for 
about ten percent of the variance. No other factors accounted for 
as much as eight percent of the variance. 

By rotating the first two factors. It was possible to force them 
to appear to be a flrst-half-of-the-test factor and a second-half-of- 
the-test factor. But this was not very obvious and a much more de- 
fensible Interpretation would be one general factor (perhaps "Intel- 
tlgence") and no other significant factors. 

This later Interpretation was further supported by a second 
principal components factor analysis which was performed after sub- 
stituting the squared multiple correlations (as an approximation for 
the communallties) In the principal diagonal of the matrix. This 
analysis also yielded one general factor which accounted for about 
forty-six percent of the variance and no other factors which accounted 
for as much as four percent of the variance. 

TABLE IV 



PRINCIPAL COMPONENTS FACTOR ANALYSIS 
WITH UNITY IN THE PRINCIPAL DIAGONAL 



FACTOP^ 



SUB- 


1 


2 


3 


4 


5 


6 


7 


8 


SCORES 


1 


.6854 


.2575 


.5215 


.0283 


.4279 


.0603 


.0354 


.0546 


2 


.6371 


.4695 


-.5293 


.0523 


.2755 


.0414 


-.0940 


-.0642 


3 


.7644 


.2741 


.0073 


.0065 


-.2547 


-.3433 


.3961 


.0245 


4 


.7864 


.1795 


.0566 


.1148 


-.3717 


.2262 


-.2321 


.2993 


5 


.8359 


-.0373 


.1466 


-.0160 


-.2105 


.0803 


-.1468 


-.4535 


6 


.6942 


-.2454 


-.1009 


-.6407 


.0493 


.1401 


.0926 


.0793 


7 


.7353 


-.4066 


-.0386 


.0692 


.1277 


-.4246 


-.2884 


.0884 


8 


.6775 


-.4807 


-.1393 


.3825 


.0973 


.2428 


.2744 


.0122 


EIGEN 


VALUE 


4.2593 


.8523 


.6080 


,S787 


.5347 


.4398 


.4095 


.3173 


PERCENT 


TRACE 


53.242 


10.653 


7.600 


7.234 


6.682 


5.497 


5.119 


3.966 
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TABLE IV 
CX)NTINUED 



VARIMAX ROTATION OF THE FIRST TITO FACTORS 



FACTORS 



SUB- 


1 


2 














SCORES 

1 


.6753 


-.2830 














2 


.7856 


-.0957 














3 


.7441 


-.3252 














4 


.6952 


-.4090 














5 


.5823 


-.6008 














6 


.3366 


-.6549 














7 


.2557 


-.8003 














8 


.1629 


-.8147 
















PRINCIPAL COMPONENTS FACTOR ANALYSIS 
WITH SQUARED MULTIPLE CORRELATIONS 
IN THE PRINCIPAL DIAGONAL 






SUB- 


FACTORS 
1 2 


3 


4 


5 


6 


7 


8 


SCORES 

1 


.6238 


.1212 


.0546 


.0682 


.0601 


.0757 


.1761 


.0786 


2 


.5730 


.1953 


-.0893 


-.1553 


.1238 


.0696 


-.0246 


-.0404 


3 


.7186 


.1839 


.0943 


.0230 


-.2498 


.0872 


-.0254 


-.0119 


4 


.7481 


.1431 


-.0342 


.2024 


.0924 


.0908 


-.0467 


-.0751 


5 


.8100 


-.0152 


-.0569 


-.2353 


-.0169 


.0983 


-.0211 


.0424 


6 


.6341 


-.1262 


.0928 


.0875 


.0678 


.0770 


-.1266 


.1062 


7 


.6870 


-.2571 


-.2625 


.0762 


-.0724 


.0834 


.0401 


-.0123 


8 


.6213 


-.2541 


.2294 


-.0625 


.0358 


.0754 


.0438 


-.0874 


EIGEN 

VALUE 


3.7102 


.2540 


.1544 


.1431 


.1013 


.0547 


.0544 


.0345 


PERCENT 

TRACE 


46.378 


3.175 


1.931 


1.788 


1.266 


.684 


.681 


.431 
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TABLE IV VARIMAX ROTATION OF THE FIRST TWO FACTORS 
CONTINUED 



FACTORS 



SUB- 1 2 

SCORES 



1 .5413 -.3329 



2 



5541 -.2440 



3 



6535 -.3509 



4 



6476 -.4009 



5 



5860 -.5594 



6 



3814 -.5221 



7 



3317 -.6542 



8 



2854 -.6075 



It would seem that the most reasonable conclusion to be drawn 
from the factor analysis is that there is one large general factor 
(probably "intelligence") which accounts for about half of the non- 
eiror variance and that there arc probably no other major factors 
which involve more than one of the subscores. This leaves about 
fifty percent of the non-error variance to be accounted for by the 
individual subscores. 

An analysis of the unique contribution of each subscore to the 
non-error variance was undertaken. The total variance or a partic- 
ular subscore may be thought of as being composed of a portion which 
is common with and may be accounted for by the rest of the test 
(this portion may be approximated by the squared multiple correlation 
of all the other subscores with the one subscore in question and 
symbolized by and a portion which is unique to the particular 

subscore (symbolized by Sui>- 

Since both of these portions are normally expressed as decimal 
proportions of the total variance, they must sum to unity. 






f 









i 



I 






I 



i 



'er|c 






UfliffiliffflffTlTliiU 



^3 

■iK 






Soi - 1 - ’^IK 



-17- 

That Is: 
or; 

This is the equation for the uncorrected unique variance. A cor- 
rection for attenuation due to error or unreliability must be ap- 
plied to obtain the corrected unique proportion of the non-error 
variance (symbolized by q®ui^* The correction Involves division 
of the squared multiple correlation by the product of the reliability 
of the subscore in question and the reliability of the rest of 



the test The formula is; « 1 - ^IK 



^il X *’EcK 

This analysis was performed on each of the eight subscores of 



Test of Science Processes and the results are presented in the 
table below. 



TABLE V 



UNIQUENESSES 



SUB- 

SCORE 




-,•3 


T * 
^11 






* itt. 


c^Ui 


I 


.5962 


1 .3554 


.42 


.91 


.3822 


.9299 


.0701 


1 

II 


.5612 


.3150 


.33 


.91 


.3003 


1.0490 


-.0490 


III 


.6910 


.4774 


.61 


.90 


.4490 


.8696 


.1304 


IV 


.7215 


.5206 


.67 


.90 


.6030 


.8633 


.1367 


V 


.7703 


.5933 


.77 


.88 


.6776 


.8756 


.1244 


VI 


.6079 


.3696 


.47 


.91 


.4277 


.8642 


.1358 


VII 


.6753 


.4561 


.59 


.90 


.5310 


.8589 


.1411 


VIII 


.6190 


.3832 


.50 


.91 


.4550 


.8422 


.1578 



* reliabilities calculated using Kuder-Richardson formula 20 



The last column on the right In Table V contains the unique pro- 
portion of the non-error variance contributed by each of the eight 
stibscores. (THie ”-.0490" reported for subscore II is probably a random 
variation from a true value of approximately zero, since a negative 
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unlqueness is meaningless.) It should be noted that the uniqueness 
Is the proportion of the non-error variance (expressed by the 
reliability, rij). So, for example, in subscore VIII, about 15 3/4 
percent of the 50 percent non-error variance Is unique to that sub- 
score. 

Several interpretations could be offered for the results of this 
analysis, but it would probably be best to wait for a more controlled 
investigation (perhaps with cross-validated reliabilities and 
alternate forms of the test). For the present, it should be suf- 
ficient to say that the last six subscores appear to have at least 
a certain degree of uniqueness and subscores I and II may not be 
very unique because their short lengths and low reliabilities may 
preclude such a possibility. 

The statistics of this uniqueness analysis are discussed in 
detail by Davis. ^ It might be noted that Davis considered unique- 
nesses of more than .12 to be quite good. 

COMPUTER PROGRAMS 

During the course of this study, several computer programs have 
been written and adapted for statistical processing of the data 
needed for test development. These include an item analysis program 
and programs for scoring, subscoring, reliability, validity, and 
norming. 

These programs incorporate several new, unique, and efficient 
features and are now available for use in the development and evalu— 
tlon of other new test materials « Among these features are: 

Iprederick B. Davis, Identification and Measurement of Reading 
Skills of High-School Students (Washington, D.C.; Office of Education, 
United States Department of Health, Education, and Welfare, Cooperative 
Research Project no. 3023, 1967). 
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A. 



In the item analysis programs, the option to call for any combin- 
ation of the followings frequency of responses to each choice 
(including "omits" and "didn’t reaches"), percent of responses, 
point biserial correlations of each choice with total score, and 
blserial correlations of each choice with total score. Also, 



g 



I 



the user has the option of concurrent scoring of each test paper 
during the analysis. 

B. In the scoring and subscoring program, the user receives Kuder- 
Richardson formula 20 reliabilities for each subscore (up to a 
maximum of ten) as well as for the total score and group means, 
standard deviations, and ninety-five percent confidence inter- 
vals for the means of each subscore as well as for the total 
score. Furthermore, the subscores may be overlapping (i.e., 
items may be included In two or more subscores simultaneously.) 

C. The correlation program used in norming and validating allows 
for from two to seventy-five variables to be correlated at one 
time all within the 60K core storage of an IBM 1620 model II 
computer. It is so flexible that only one format card needs 

to be changed for any one run even though the number of variables 



changes. It also allows the user to obtain the regression 
equations for all or some of the variables merely by setting the 



console switches 






EDUCATIONAL SIGNIFICANCE OF THE STUDY 




1. Need ; Current trends in science education have created a 
need for Instruments with which to measure students* progress toward 
newly defined objectives. Until the completion of the Test of 
Science Processes , no instrument was available with which to measure 
seventh, eighth, and ninth grade students' ability to use science 
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processes. This study was an attempt to fill this need. The In- 
strument Is Intended for making measurements of progress toward 
behavorlal objectives which stress science processes. It should 
enable educators to prepare more reliable and valid evaluations of 
their students' achievement In these areas which are now being empha- 
sized. 

Extent of Applicability ; In view of the basic nature of 
the processes being tested and the non-specific nature of the content 
material employeds this Instrument should be applicable for use with 
almost all students In grades seven, eight, and nine throughout the 
country. As Jacobson has so eloquently stated: "Science programs 

must do much more than prepare young people for science research; 
every young American, regardless of fels future calling, has a right 
to an education In science that will be of Interest and value. 

In order to make the Instrument available to the largest number of 
students (and more valid at the same time) an attempt has been made 
to minimize the reading difficulty of the Items, but never at the 
expense of clarity or scientific accuracy. Questions Include math- 
ematics because science Is mathematical; however, the computations 
have been kept as simple as possible so that the students who are 
weak In mathematics are not unduly penalized. 

3. Diagnosis; Because of the specific, item-by-item analysis 
provided in the test manual, and the subscoring routines and norms, 
the Test of Science Processes makes possible much more specific analyses 
of the weaknesses of curricula than was previously possible. Also, it 

^Willard J. Jacobson, "Science Curriculum Change in the United 
States sines 1957," a draft of a chapter from the forthcoming book. 
Strategies for Curriculum Change , edited by R. Murray Thomas, Lester 
B. Sands, and Dale L. Brubaker (mimeographed, 1967) p. 21. 
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may now be possible to prescribe remedial work for students and place 
them more efficiently in their future courses - this remains to be 
investigated and can only be regarded as tentative in the light of 
the low rellabllltes of some of the subscores. 

There is an alarmingly high drop-out rate in high school science 

courses. It has been reported that about seventy-seven percent of 

all junior high school students are enrolled in some science course.^ 

When these students reach high school, they are usually required to 

take biology, and it is reported that eighty-six percent of the 

students in the grade in which biology is offered are enrolled in a 

2 

biology course. By the time these students reach the grade in which 
chemistry is offered, only thirty-eight percent of them are enrolled in 
the course. And, by the time they reach the grade in which physics 
is offered, only twenty-three percent of them are enrolled in the 
physics course.^ 

One possible explanation for at least part of this tremendous 
drop-out rate in science is that students have specific weaknesses in 
various areas required in high school science courses and because of 
these they have failed to enroll in or have dropped out of science 
courses. If the Test of Science Processes makes possible a better 

1 

Lola Ericksen Rogers, Science Teaching in the Public Junior 
High School (Washington, D.C .5 United States Government Printing 
Office, 1967) (Catalogue number 5.5229:29067) p. 8 . 

2 

Kenneth A. Simon and W. Vance Grant, Digest of Educational 
Statistics 1966 (Washington, D.C.; United States Government Printing 
Office, 1966) (Catalogue number 5.210:10024-66) p. 34. 

%id . 

^Ibid. 
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dlagnosls of some of these weaknesses, then, perhaps, remedial work, 
assigned on the basis of the diagnosis, may contribute to a decrease 
in the science drop-out rate» 

4, Use in Educational Research ; Because of the unique nature of 
the test and its very complete manual, the Test of Science Processes 
should be a valuable tool for educational researchers. For example, 
using the test, the differences in students resulting from various 
types of teaching could be studied. Or, other Instruments of a 
similar nature could be developed and validated. Or, any of numerous 
other research projects could be undertaken which investigate '*objec— 
tively” areas previously measurable only in terms of observations or 
subjective reports. In fact, the very existence of an Instrument of 
this nature (i.e., objective testing of process-related behavioral 
objectives) demonstrates to science educators the feasibility of the 
development of other such Instruments and of objective studies in 
this area. 
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APPENDIX A 

LIST OF SCIENCE EDUCATION EXPERTS TO VniOM THE STATEMENT 
OF SCIENCE PROCESSES WAS SENT FOR VALIDATION AND COMMENT 



1. 


J. Myron Atkin 


16. 


Vernon Rockcastle 


2. 


Darrell Barnard 


17. 


Leopold Klopfer 


3. 


Glenn Blough 


18. 


George Pallrand 


4. 


David Butts 


19. 


Archie Lacey 


5. 


Albert Carr 


20. 


Gladys Klelnman 


6. 


Robert Karplus 


21. 


Steven Winter 


7. 


Paul Rosenbloom 


a 

CM 

CM 


Charles Walcott 


8. 


Herbert Schwartz 


23. 


John Read 


9. 


Harold Splelman 


24. 


Paul Hurd 


10. 


Herbert Thler 


25. 


Robert Gagne 


11. 


Jay W. Erickson 


26. 


Ernest Burkman 


12. 


Henry Walbesser 


27. 


W. C. Van Deventer 


13. 


Harry V. Scott 


28. 


Frederick Ferris 


14. 


Fletcher Watson 


29. 


Uri Haber-shaim 


15. 


Matthew Bruce 


30. 


Charles Helmler 






31. 


John H. Wocdburn 



The Inclusion of a science educator on this list does not In any 
way Imply his agreement with or endorsement of the Test of Science 
Processes or Its blueprint. As Is Indicated In the text, not all of 
these people returned the statement which was sent to them and It Is 
Impossible to determine which ones returned It because they were not 



necessarily signed. 

In order to be Included on the list of people to whom the state- 
ment of science processes was mailed, a person had to meet requirements 
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one and two below and either three or four or both three and four. 
Requirements : 



MUST 

HAVE 



EITHER 
OR BOTH 



ri. Has taught children at elementary and/or 
junior high school level and/or prepared 
teachers to teach at either one or both 
of these levels. 

4 / 2. Has been recommended by at least one mem- 

ber of professorial rank of the Department 
of Science Education of Teachers College, 
Columbia University as being knowledgeable 
regarding developments In elementary and 
. junior high school science education. 



3. Has published or done research on science 
for the junior high school. 




Has worked with a recent national science 
curriculum project (since 1960.) 



The respondents were given the following Instructions: "For each 



of the behaviors listed under the eight processes, please circle the 



appropriate number under each category. The definition of the cate- 
gories is given below. Please feel free to make any written comments 
wherever you feel they are needed. 

- UNCLEAR - needs major rewriting. 



CLARITY: 




2 - CLEAR, but the minor changes Indicated 

would be very clarifying. 

3 - CLEAR AS WRITTEN - no changes needed. 



f 1 - INAITEOPPJATE - not wortl. including. 
APPROPRIATENESS IN \ 

THE LIGHT OF CURRENT < 2 - APPROPRIATE, but not of the utmost Importance. 
SCIENCE EDUCATION: 



3 - CRUCIAL - must be Included. 



DIFFICULTY WITH 
REGARD TO THIS 
AGE LEVEL: 



- VERY EASY for 11- to 14-year-old students — 
trivial when asked of this age 
student. 

^ 2 - TOO DIFFICULT for most 11- to 14-year-old 

students — do not Include. 




3 



APPROPRIATE to expect students at this age 
level to know or to be learning. 



ft 






-25- 

APPENDIX B 
BIBLIOGBAPHY 



This appendix consists of a bibliography of those materials to 
which reference was made during the construction of the Test of Science 
Processes and its manual* * Those entries which are narked v;ith a 

are general references which were not consulted during the preliminary 
research for and construction of the blueprint. All of the other 
entries were consulted at least once during either the preliminary 
research for or construction of the blueprint and may have been referred 
to at other times, also. 
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Process Approach . Washington: American Association for the 
Advancement of Science, 1965. (aAAS Miscellaneous Publication 65-8.) 

American Psychological Association. Standards for Educational and 

* Psychological Tests and Manuals . Washington: American Psycholog- 
ical Association, Inc., 1966. 

Ames, Maurice U., Arthur 0. Baker and Joseph F. Leahy. Science for 

Progress . Englewood Cliffs, New Jersey: Prentice-Hall Co.. 1956. 

Aylesworth, Thomas G. Planning for Effective Science Teaching . 

Columbus, Ohio; American Education Publications, Education Center » 
1963. 

Barnard, J. Darrell. Teaching High-School Scie nce. Washington: 

National Education Association, 1956. 

Barnard, J. Darrell, Celia Steadier, Benjamin Spock and Nelson F. Beeler. 
Science: A Search for Evidence . New York: Macmillan Company, 1960. 

Barnard, J. Darrell, Celia Steadier, Benjamin Spock and Nelson F. Beeler. 
Science: A Way to Solve Problems . New York: Macmillan Co., 1960. 
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APPEWDIX C 



BLUEPRINT OF THE TEST OF SCIENCE PROCESSES 



Process I 



OBSERVING 



In order for a student to demonstrate competence in using the process 



of observing, he should be able to do the following; 



Behaviors 



Questions 



2 

1. Demonstrate an operational knowledge^ of the 



physical properties of objects. 

2. Identify and describe the objects which 
interact in a system. 



14. 19 



13, 18 



3. Identify and describe the results of inter- 



actions of objects and systems of objects 



in terms of Initial and final states. 



17 



4. Distinguish among various spatial relationships 



of the objects within a given system. 

5. List the observable characteristics of a given 



15, 20 



phenomenon. 



16, 21 



^The numbers under this heading are the numbers of the questions 

on Form II of the Test of Science Processes which are intended 
to assess the students ' ability to perform the specified behavior. 



^The terns and phrases used in the blueprint which do not have 
precise behavioral or scientific meanings, or whose common meanings 
are either Imprecise or somewhat different from their usage in the 
blueprint, are defined in the "Glossary" which follows Process VIII, 
Predicting. 
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Process II COMPARING 

In order for e student to demonstrete competence In 
of comparing, he should be able to do the following: 
Behaviors 

1* Describe, In terms of their physical properties 
the similarities of two or more of each of the 
following : 



using the process 

Questions 
3, 23, 24 



a. Objects 

b. Systems of objects 

c. Interactions of objects and of systems of 
objects 

d. Relative positions of objects 

2. Contrast, on the basis of differences In 7, 22 

their physical properties, two or more of each 
of the following: 

a. Objects 

b. Systems of objects 

c. Interactions of objects and of systems 
of objects 

d. Relative positions of objects 



Process III CLASSIFYING 

In order for a student to demonstrate competence In using the 
process of classifying, he should be able to do the following: 
B ehaviors Questions 



1. Group objects or systems of objects according 
to a given property. 



9, 10, 26 
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Process III CLASSiraNG (cont.) 



Questions 



2. Select and justify an appropriate property 



1. 27 



and group objects or systems of objects 



according to that property 



3. Group objects or systems of objects according 



4, 5 



to two simultaneous properties 



4. Select and justify two or more appropriate 



28. 29 



simultaneous properties and group objects or 
systems of objects according to these properties. 

5. Given a group of objects, Identify the property 25, 11 
or properties on which they are grouped. 



remove a specified number of members from the 
original set to form two new sets which are 
grouped on the basis of one or more given simul- 
taneous properties. 

Process IV QUANTIFYING 

In order for a student to demonstrate competence In using the 
process of quantifying, he should be able to do the following; 
Behaviors Questions 

1. Demonstrate an operational knowledge of 30, 32, 36 

ordinal and cardinal numbers up to one 

million and of negative numbers. 

2. Demonstrate an operational knowledge of 33, 35, 37 

simple fractions, percents, and decimals. 



6. Given a set of objects or systems of objects 



6 , 2 
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Process IV QUANTIFYING (cont.) 

3. Be able to order a group of objects or 
systems of objects from most to least (or 
vice-versa) on one or more simultaneous 
orderable properties. 

4. Be able to arrange and to read data In 
various graphic and t^ular formats. 



Questions 
31, 34 



38, 39 

40, 41 
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Process V MEASURING i 

1 



In order for a student to demonstrate competence in using the process 



of measuring, he should be able to do the following: 
Behaviors 

1. Suggest and use ’’home-made*' units for 
measuring the properties of objects. 

2. Demonstrate an operational knowledge of units 
of measure, the function of widely accepted 



Questions 

51, 53 

52, 54 

62, 65 



units, the names and approximate sizes of the 66 
most common units such as Inch, foot, centimeter, 
meter, pound, quart, gram, kilogram, liter, 
second, degree Celsius, etc. 

3. Be able to select the appropriate units for 46, 60 

making a particular measurement. 



4. Be able to estimate the dimensions and proper- 47, 50 
ties of in object (Including temperature, 

size, mass, etc.) for purposes of ordering, 
describing, and classifying. 

5. Be able to measure the dimensions and properties 45, 55 

of an object (Including temperature, size, mass, etc.) 
for purposes of ordering, describing, and classifying. 
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Process V MEASt^RING (cont.) 



Questions 



6* Demonstrate an operational knowledge of area 



56, 57 



and volume In terms of one-, two-, and three- 



A 

dimensional measurements (e.g., a«l and a»l x w; 



3 2 

v*l I v*h X a*h X 1 5 and v«h x 1 x w.) 



7. Be able to measure time. 



44, 58 



8. Be able to measure the rate of change of a 



48, 59 



property of an object or a system of objects. 
9. Represent and recognize an object or a system 



49, 61 



of objects by a scale diagram. 



10. Represent and recognize the spatial relation- 63, 64 



ships among two or more objects by a scale 
diagram (mapping.) 



11 . Recognl ze the appropriateness and limitations 



42, 43 



of measuring devices In a given situation. 



Process VI EXPERIMENTING 



In order for a student to demonstrate competence In using the process 
of experimenting, he should be able to do the following: 

Use suitable experimental procedures In seeking solutions to problems 



Including possibly: 
Behaviors 



Questions 



1. Design an Investigation appropriate to the 
problem: 



a. Select, clarify, and state In testable 
terms (perhaps as an answerable question) the 
primary variable to be Investigated. 



67, 71 



b. Control the variables appropriately so that 



68 



logical conclusions may be drawn with regard to 
the primary variable. 
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Process VI EXPERIMENTING (cont.) QuestloM 

c. Distinguish between dependent and 69, 72 

independent variables . 

2. Perform the investigation; 

a. Design, construct, or select, and success- 70, 76 

fully utilize apparatus to assist in data 

gathering, where appropriate. 

b. Employ the processes of observing, comparing s 
quantifying, classifying, and measuring to gather data.^ 

c. Repeat the data gathering a sufficient number 75, 77 

of times to improve reliability and under sufficiently 
varying conditions to account for the Influence of 
different variables. 

d. Record and organize the data gathered in a logical form.^ 

3. Utilize the processes of inferring and predicting to 74 

Interpret the data collected, answer the original 
problem, and lead to the posing of new problems and 
the design of new escperiments to investigate them. 

Process VII INFERRING 

In order for a student to demonstrate competence in using the process 
of inferring, he should be able to do the following; 

Questions 

Behawiors 

1. Draw warranted generalizations from a body of data. 78, 82 



I ^There are no questions specifically designed to assess this behavior 

I (because it does not lend itself to short-answer questions) however, it 

I • has been included for logical continuity and completeness in the 

t blueprint. 
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Process VII IHFE2RING <C0nt.) 



Questions 



2. Identify the factor most likely to have caused 92, 95 



a given change In a system. 

3. Identify and specif 3 ' observations which would 



94, 96 



be needed to justify a particular generalization. 

4. Be able to distinguish between a statement 12, 79 

based directly on observations and one which Is 

an Inference or a generalization. 

5. Be able to draw more than one Inference in 73, 85 



situations where the data allows^ this. 



6 . Be able to test an Inference by collecting 



80, 83 



further data. 

7. Recognize which data are necessary and sufficient 81, 86 

to support an Inference or a generalization. 



Process VIII PREDICTING 



In order for a student to demonstrate competence in using the process 



of predicting, he should be able to do the following: 

Behaviors Quest lon£i 



1. Be able to detect or demonstrate trends in data 88 , 89 



(presented in many different ways) and be able to 
use these trends to predict by extrapolation 



and/or Interpolation. 

2. Devise and use simple means of checking the 



84, 90 



accuracy of the predictions made. 


















Process VIII PREDICTING (cent.) Questions 

3. Recognize and use pertinent arguments, reasons, 91, 93 

or principles to justify a prediction. 

4. Demonstrate an operational knowledge of the 8, 87 

necessity for multiple and reliable obser-» 

vatlons prior to prediction and an unwilling- 
ness to offer predictions In the absence of 
such observations. 



% 



% 












iErIc 









iillil 



> -••...., ......... , .... 







I 







I 

?■ 



I 



i 



i. 

?: 

{• 

t 



- 39 - 

GLOSSARY 

The terms and phrases used in the blueprint which do not have 
precise behavioral or scientific meanings, or whose common meanings 
are either imprecise or somewhat different from their usage in the 
blueprint, are defined as follows: 

Factors ; For example, gravity, physical properties, spatial 
relationships, heat, light, magnetism, electrical forces. 

Interaction ; The process by which two or more objects or forces 
or systems influence each other causing changes in one or more of the 
Interactants . 

Operational knowledge ; The ability to use the concepts, constructs, 
etc. under consideration without necessarily having command of the 
technical vocabulary associated therewith. 

Physical Properties ; For example, phase, motion, color, mass, 
length, volume, area, density, temperature. 

Simultaneous properties ; Properties which are to be considered 
together. For example, objects which are at the same time red and 
square versus objects which are not at the same time red and square. 

System ; An assemblage of objects and factors which affect and/or 
characterize those objects, arbitrarily selected for study from the 
totality of objects and factors. 
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